AI capability research and development platform and data processing method

ABSTRACT

Embodiments of the present disclosure provide an AI capability research and development platform and a data processing method. The AI capability research and development platform includes: a data management module, a tool management module, a process management module and a model management module, where the data management module is configured to perform data processing on received data, including at least one of the following: analyzing data type of the data, converting the data according to preset data format and storing the data; the tool management module is configured to store at least one tool, each tool being used to execute a preset processing flow; the process management module is configured to perform model training according to the tool provided by the tool management module and the data provided by the data management module; the model management module is configured to store a model obtained by the model training.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No.201910591585.1, filed on Jul. 2, 2019, which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of communicationtechnologies and, in particular, relates to an AI capability researchand development platform and a data processing method.

BACKGROUND

When realizing the development of a model, six links are usually needed,including: data collection, data annotation, model training, testing,launching, and encapsulation and calling.

In the prior art, these six steps are completed offline in sixindependent links. For example, data collection needs the business partyor the strategy developer to perform targeted collection, and needs tobe independently managed by offline individuals; the data annotationneeds the strategy developer to present requirements to the publictesting, and offline docking needs to be performed for the annotationcontent; the model training needs to be completed independently by thestrategy developer offline, and related resources needs to be managed;the testing needs to be completed by docking testers offline; thelaunching needs to be completed by docking architecture developersoffline; and the encapsulation and calling needs the external businesspart to perform encapsulation to obtain an externally callable service.

However, in the prior art, when performing the six steps, offlinepersonnel from multiple parties need to communicate and debug with eachother, resulting in a very low development efficiency.

SUMMARY

Embodiments of the present disclosure provide an AI capability researchand development platform and a data processing method to solve thetechnical problem of low efficiency in model development in the priorart.

A first aspect of embodiments of the present disclosure provides an AIcapability research and development platform, including:

a data management module, configured to perform data processing onreceived data, where the data processing includes at least one of thefollowing: analyzing a data type of the data, converting the dataaccording to a preset data format, and storing the data;

a tool management module, configured to store at least one tool, each ofthe at least one tool being used to execute a preset processing flow;

a process management module, configured to perform model trainingaccording to the tool provided by the tool management module and thedata provided by the data management module; and

a model management module, configured to store a model obtained by themodel training.

Optionally, the data management module is further configured to:

collect to-be-back-flow data, where the to-be-back-flow data is datathat meets a preset back-flow condition; the to-be-back-flow data isused to provide source data for model iteration.

Optionally, collecting to-be-back-flow data includes:

setting the to-be-back-flow data in a back-flow catalog; and

collating the back-flow catalog according to a preset frequency; wherethe collating includes: sorting to-be-back-flow data which is collectedduring a preset time period, and setting a same kind of to-be-back-flowdata in the to-be-back-flow data which is collected during the presettime period into one back-flow catalog.

Optionally, the tool management module is further configured to:

receive, in a tool creation page of the tool management module, a toolcreation operation of a user; and

generate a tool according to the tool creation operation.

Optionally, the platform further includes:

a test module, configured to test the model obtained by the training;and

a platform management module, configured to coordinate and manage thedata management module, the tool management module, the processmanagement module, the model management module and the test module at aproject granularity.

Optionally, the test module is further configured to: generate a testreport.

Optionally, the module management module is further configured to storeat least one of a creator, training data set information and model indexinformation of the model.

Optionally, a type of the tool includes at least one of the following: adata cleaning type, a data mining type, a model training type, a serviceevaluation type, a data back-flow type, and a batch prediction type.

Optionally, the data management module is specifically configured to:

receive to-be-processed data of a project;

analyze a data type of the to-be-processed data;

convert to-be-processed data whose data type meets a preset condition inthe to-be-processed data into target data; where the target data has thepreset data format;

perform statistic on the target data; and

sort the target data into a test set and a data set.

Optionally, the data management module is further configured to:

modify, according to a modification operation of the user,to-be-processed data whose data type does not meet the preset conditionin the to-be-processed data into data of the preset data format.

Optionally, performing statistic on the target data includes:

querying data of at least one preset class in the target data; and

performing, in each of the at least one preset class, statistic on dataof the each of the at least one preset class.

Optionally, the process management module is further configured to:

provide a model training user interface;

receive, in the model training user interface, a target data set and atarget tool selected by the user;

receive, in the model training user interface, a connecting line betweenthe target data set and the target tool from the user;

transfer the target data set to the target tool according to theconnecting line; and

perform model training according to the target tool to obtain a trainedmodel in a case of receiving a running instruction.

Optionally, the process management module is further configured to:

display a model corresponding to a model viewing instruction in a caseof receiving the model viewing instruction.

A second aspect of embodiments of the present disclosure provides a dataprocessing method, which is applied to the AI capability research anddevelopment platform described in the first aspect of the embodiments ofthe present disclosure, where the method includes:

performing data processing on received data, where the data processingincludes at least one of the following: analyzing a data type of thedata, converting the data according to a preset data format, and storingthe data;

performing model training according to a tool and the data; and

storing a model obtained by the model training.

Optionally, the method also includes:

collecting to-be-back-flow data, where the to-be-back-flow data is datathat meets a preset back-flow condition; the to-be-back-flow data isused to provide source data for model iteration.

Optionally, the collecting to-be-back-flow data includes:

setting the to-be-back-flow data in a back-flow catalog; and

collating the back-flow catalog according to a preset frequency; wherethe collating includes: sorting to-be-back-flow data which is collectedduring a preset time period, and setting a same kind of to-be-back-flowdata in the to-be-back-flow data which is collected during the presettime period into one back-flow catalog.

Optionally, the method also includes:

receiving, in a tool creation page, a tool creation operation of a user;and

generating a tool according to the tool creation operation.

Optionally, the method also includes:

testing the model obtained by the training.

Optionally, after testing the model obtained by the training, the methodfurther includes:

generating a test report.

Optionally, a type of the tool includes at least one of the following: adata cleaning type, a data mining type, a model training type, a serviceevaluation type, a data back-flow type, and a batch prediction type.

Optionally, the performing data processing on received data includes:

receiving to-be-processed data of a project;

analyzing a data type of the to-be-processed data;

converting to-be-processed data whose data type meets a preset conditionin the to-be-processed data into target data; where the target data hasthe preset data format;

performing statistic on the target data; and

sorting the target data into a test set and a data set.

Optionally, the method also includes:

modifying, according to a modification operation of the user,to-be-processed data whose data type does not meet the preset conditionin the to-be-processed data into data of the preset data format.

Optionally, the performing statistic on the target data includes:

querying data of at least one preset class in the target data; and

performing, in each of the at least one preset class, statistic on dataof the each of the at least one preset class.

Optionally, the method also includes:

providing a model training user interface;

receiving, in the model training user interface, a target data set and atarget tool selected by the user;

receiving, in the model training user interface, a connecting linebetween the target data set and the target tool from the user;

transferring the target data set to the target tool according to theconnecting line; and

performing model training according to the target tool to obtain atrained model in a case of receiving a running instruction.

Optionally, the method also includes:

displaying a model corresponding to a model viewing instruction in acase of receiving the model viewing instruction.

A third aspect of embodiments of the present disclosure provides anelectronic device, including: a processor, a memory, and a computerprogram; where the computer program is stored in the memory andconfigured to be executed by the processor, and the computer programincludes instructions for executing the method as described in any oneof the foregoing second aspect.

A fourth aspect of embodiments of the present disclosure provides acomputer-readable storage medium, having a computer program storedthereon, which, when being executed, implements the method as describedin any one of the foregoing second aspect.

Advantages of the embodiments of the present disclosure compared withthe prior art are as follows:

the embodiments of the present disclosure provide an AI capabilityresearch and development platform and a data processing method, wherethe AI capability research and development platform can support theonline process of data collection and model acquisition, and canefficiently perform model development. Specifically, the AI capabilityresearch and development platform of the embodiments of the presentdisclosure includes a data management module, a tool management module,a process management module and a model management module, where thedata management module is configured to perform data processing on thereceived data, where the data processing includes at least one of thefollowing: analyzing the data type of the data, converting the dataaccording to the preset data format and storing the data; the toolmanagement module is configured to store at least one tool, each toolbeing used to execute a preset processing flow; the process managementmodule is configured to perform model training according to the toolprovided by the tool management module and the data provided by the datamanagement module; the model management module is configured to storethe model obtained by the model training. During model training in theembodiment of the present disclosure, data collection, model trainingand the like can be uniformly processed on the platform withoutcommunication and debugging by offline personnel from multiple parties,and the development efficiency is relatively high.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to illustrate technical solutions in embodiments of the presentinvention or in the prior art more clearly, the drawings used indescription of the embodiments or the prior art will be brieflydescribed below. Obviously, the drawings in the following descriptionare merely some embodiments of the present invention, and other drawingsmay be obtained by those skilled in the art according to these drawingswithout any creative effort.

FIG. 1 is a schematic diagram of functional modules of an AI capabilityresearch and development platform according to an embodiment of thepresent disclosure;

FIG. 2 is a schematic diagram of a data list interface of a datamanagement module according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a data detail interface of a datamanagement module according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an interface of a tool managementmodule according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a sketchpad interface of a processmanagement module according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an interface of a model managementmodule according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of an interface of a task module accordingto an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a back-flow catalog of an AI capabilityresearch and development platform according to an embodiment of thepresent disclosure;

FIG. 9 is schematic diagram of a target data display interface in an AIcapability research and development platform according to an embodimentof the present disclosure;

FIG. 10 is a schematic diagram of a split data set interface in an AIcapability research and development platform according to an embodimentof the present disclosure;

FIG. 11 is a schematic diagram of a sketchpad page interface in an AIcapability research and development platform according to an embodimentof the present disclosure;

FIG. 12 is a schematic diagram of a log interface in an AI capabilityresearch and development platform according to an embodiment of thepresent disclosure; and

FIG. 13 is a schematic flowchart of data processing according to anembodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The technical solutions in embodiments of the present disclosure will bedescribed in the following with reference to the accompanying drawingsof the embodiments of the present disclosure. Obviously, the describedembodiments are merely part of embodiments of the disclosure, not allembodiments. Based on the embodiments of the present disclosure, allother embodiments obtained by those skilled in the art without creativeeffort shall belong to the protection scope of the present disclosure.

Exemplary embodiments will be described in detail here, examples ofwhich are shown in the accompanying drawings. When the followingdescription refers to the accompanying drawings, the same numerals indifferent drawings represent the same or similar elements unlessotherwise indicated. The implementations described in the followingexemplary embodiments do not represent all implementations consistentwith the present disclosure. On the contrary, they are only examples ofapparatus and methods that are consistent with some aspects of thedisclosure and detailed in the appended claims.

It should be clear that the described embodiments are merely part of theembodiments of the present disclosure, not all embodiments. Based on theembodiments of the present disclosure, all other embodiments obtained bythose skilled in the art without creative effort shall belong to theprotection scope of the present disclosure.

The terms used in the embodiments of the present disclosure are for thepurpose of describing particular embodiments merely and are not intendedto limit the present disclosure. The singular forms “a”, “said”, and“the” used in the embodiments of the present disclosure and the appendedclaims are also intended to include the plural forms, unless the contextclearly indicates other meanings.

It should be understood that the term “and/or” used herein is merely anassociation relationship describing associated objects, which means thatthere may be three kinds of relationships, for example, A and/or B, mayindicate three situations: A exists alone, A and B exist at the sametime, B exists alone. In addition, the symbol “I” herein generallyindicates that the related objects before and after the symbol are in an“or” relationship.

Depending on the context, the words “as if”, “if” used here may beinterpreted as “at” or “when” or “in response to determining” or “inresponse to detecting”. Similarly, depending on the context, the phrases“if it is determined” or “if it is detected (stated condition or event)”may be interpreted as “when it is determined” or “in response todetermining” or “when it is detected (stated condition or event)” or “inresponse to detecting (stated condition or event)”.

It should also be noted that the terms “include”, “comprise”, or anyother variation thereof are intended to cover non-exclusive inclusions,so that a commodity or system that includes a series of elementsincludes not only those elements, but also other elements that are notexplicitly listed, or elements that are inherent to this commodity orsystem. Without more restrictions, the element limited by the sentence“include one . . . ” does not exclude the existence of other identicalelements in the commodity or system including this element.

Embodiments of the present disclosure provide an AI capability researchand development platform and a method, where the AI capability researchand development platform can support an online process from datacollection to model acquisition, and can efficiently perform modeldevelopment. Specifically, the AI capability research and developmentplatform of an embodiment of the present disclosure includes: a datamanagement module, a tool management module, a process management moduleand a model management module, where the data management module isconfigured to perform data processing on received data, where the dataprocessing includes at least one of the following: analyzing a data typeof the data, converting the data according to a preset data format andstoring the data; the tool management module is configured to store atleast one tool, each tool being used to execute a preset processingflow; the process management module is configured to perform modeltraining according to the tool provided by the tool management moduleand the data provided by the data management module; the modelmanagement module is configured to store a model obtained by the modeltraining. During model training in the embodiment of the presentdisclosure, data collection, model training and the like can beuniformly processed on the platform without communication and debuggingby offline personnel from multiple parties, and the developmentefficiency is relatively high.

The AI capability research and development platform described in theembodiment of the present disclosure can be applied to a terminal, andthe terminal may include a mobile phone, a tablet computer, a notebookcomputer, a desktop computer, or a server, and other electronic devicesthat can run the AI capability research and development platform.

The AI capability research and development platform described in theembodiment of the present disclosure can be an integrated platform forproviding service support for various artificial intelligence (AI)services. In an example, the AI capability research and developmentplatform can provide data and tool support for various links of AIresearch and development; in the AI capability research and developmentplatform, data can be seamlessly converted into a machine learning modelthat can provide a predictive service; and automation support can beprovided for iterative closed loop of service effects. In an example, AIservices may include services such as pictureviolence-terrorism/pornography recognition, character recognition,large-scale classification, picture clustering and deduplication.

As shown in FIG. 1 , FIG. 1 is a schematic diagram of functionalstructures of an AI capability research and development platformaccording to an embodiment of the present disclosure. The AI capabilityresearch and development platform of the embodiment of the presentdisclosure can include:

a data management module 110, configured to perform data processing onreceived data, the data processing includes at least one of thefollowing: analyzing the data type of the data, converting the dataaccording to a preset data format and storing the data; a toolmanagement module 120, configured to store at least one tool, each toolbeing used to execute a preset processing flow; a process managementmodule 130, configured to perform model training according to the toolprovided by the tool management module and the data provided by the datamanagement module; a model management module 140, configured to store amodel obtained by the model training.

In the embodiment of the present disclosure, the data management module110 can also be referred to as a data warehouse module. Different fromthe data warehouse in general sense, which is only used to provide datastorage, the data management module 110 in the embodiment of the presentdisclosure can support convenient management of data in various formatsby a user.

Specifically, the data management module 110 can set an entry forreceiving data, and the user can upload data through the entry. Afterreceiving the data, the data management module 110 can automaticallyanalyze the data type of the data and convert the data of the expecteddata type into the preset data format, so that the user can obtain datain a unified format through the data management module 110, which avoidsthe phenomena such as low efficiency of data processing caused byinconsistent data formats.

It can be understood that the data management module 110 can alsoanalyze the data type of the data according to a user trigger. In anexample, the data management module 110 can set an analyzing control;when the user clicks the analyzing control, the data management module110 analyzes the data type of the data in response to a click operation,and further converts the data of the expected data type into the presetdata format.

In practical applications, if the data management module 110 analyzesthe data and finds that the data type of the data does not meet theexpectation, the data management module 110 can modify a configurationitem of the data according to a modification operation of the user tomake the modified data type meet the expectation, and further convertthe data of the expected data type into the preset data format, which isnot specifically limited in the embodiment of the present disclosure.

In specific applications, when performing model training, it may benecessary to perform model training using annotated sample data, thenthe user can obtain, from the data management module 110, source datathat needs to be annotated, and upload the source data to the datamanagement module 110 after annotating the source data. The datamanagement module 110 can store the annotated data in a unified mannerfor subsequent model training.

In practical applications, the function of the data management modulecan also be set according to a practical application scenario. In anexample, the data management module 110 can be set to support convenientimage data import and export as well as standardization capability, suchas supporting the hyper text transport protocol (HTTP), hadoopdistributed file system (HDFS) and other types of data sources, as wellas data standardization for compressed package format (zip), javascriptobject notation (JSON), serialization data structure protocol“protobuf”, comma-separated values (CSV) and other file formats.Specifically, the standardization is to analyze the data, extract thestructures of data samples, and store them in a standardized way forsubsequent use.

In an example, the data management module 110 can also be set to supportflexible and convenient data preview, query, and statisticscapabilities. Specifically, in order to support data of any scale, adistributed file system can be adopted at the bottom of the platform,which can meet needs of a large amount of batch data reading and writingand batch data processing. In order to meet the needs of a small amountof data sequence preview and fast query, key-value (KV) storage can beadopted in the data management module to facilitate data indexing.

In an example, the data management module 110 can also be set to supportfunctions such as data cleaning and data annotation docking, which isnot specifically limited in the embodiment of the present disclosure.

In an example, FIG. 2 is a schematic diagram of a data list interface ofthe data management module 110 according to an embodiment of the presentdisclosure. In the data management module 110, standardized data,non-standardized data, annotated data and the like can be stored. In thedata list interface of the data management module 110, the name,creation time, file type, status, sharing status and the like of eachdata set can also be displayed, which is not specifically limited in theembodiment of the present disclosure.

In an example, FIG. 3 is a schematic diagram of a data detail interfaceof the data management module 110 according to an embodiment of thepresent disclosure. In the data management module 110, operations suchas data combination, data connection, data splitting, data conversion,and data export can be performed on the data sets. In the data detailsof the data management module 110, controls corresponding to theoperations can also be displayed, including for example a datacombination control, a data connection control, a data splittingcontrol, a data conversion control, and a data export control, etc.,which is not specifically limited in the embodiment of the presentdisclosure.

In the embodiment of the present disclosure, the tool management module120 can also become a tool warehouse. One or more tools can be stored inthe tool management module 120. Specifically, a tool can be code forcompleting a certain function, and the code may specifically be acombination of multiple script files or binary files. After the tool isencapsulated and integrated into the AI capability research anddevelopment platform, the tool can be conveniently called and used. Inspecific applications, a preset processing flow in the code can beexecuted when the tool is run.

It can be understood that in practical applications, the tool managementmodule 120 can also support operations such as tool upgrade andmodification, which is not specifically limited in the embodiment of thepresent disclosure.

In specific applications, the tools in the tool management module 120can include not only tools that are fixedly set by the platform, butalso tools that are customized according to practical applicationscenarios, that is, different types of tools can be set according todifferent needs. In an example, from the types of capabilities, thetools can be divided into following types: a data cleaning type, a datamining type, a model training type, a service evaluation type, a databack-flow type, a batch prediction type, etc.; from the manners ofrunning, the tools can be divided into: a distributed-system foundationarchitecture “hadoop” tool, a universal parallel framework “spark” tool,an open source deep learning platform “paddle” cluster training tool,etc. The specific content and form of the tool are not limited in theembodiment of the present disclosure.

In practical applications, the user can define a metafile, declare aninput and output (such as input data, an output model), running resourceneeds, an execution entry and configurable parameters of a tool in themetafile. The source file and other code files are packaged and thensubmitted to the platform for release, and then the tool can execute thecorresponding process according to the configuration parameters.Optionally, the configurable parameters of the tool can be presented tothe user in a workflow page in a visual interface, so that the use ofthe tool is visualized to the user.

Usually a tool requires certain resources for its execution. In anexample, an image clustering tool requires “hadoop” resources, the modeltraining requires graphics processing unit (GPU) cluster resources, andthe batch prediction requires central processing unit (CPU) clusterresources. Through declaring the resource needs in the metafile by theuser, the tool can be directly used without coordinating variousresources.

In specific applications, the tool management module 120 can also be setaccording to actual needs to support platform-level tools andproject-level tools. The specific platform-level tools are provided bythe platform side and can be used by all users. The project-level toolscan be shared within a project, between projects and across the platformaccording to different sharing methods of users, and other users who cansee the tools can reuse them, thereby improving the efficiency of toolreuse. The embodiment of the present disclosure does not specificallylimit this.

In an example, FIG. 4 is a schematic diagram of an interface of the toolmanagement module 120 according to an embodiment of the presentdisclosure. In the tool management module 120, system tools, customtools and the like can be stored. In the user interface of the toolmanagement module 120, the name, creation time, creator, status and thelike of each tool can also be displayed, which is not limited in theembodiment of the present disclosure.

In the embodiment of the present disclosure, the process managementmodule 130 can also be referred to as a process platform. The processmanagement module can be built based on the modules provided by the AIcapability research and development platform such as the data managementmodule 110, the tool management module 120 and the model managementmodule 140. By constructing a directed acyclic graph (DAG) process, thedata, model, tools and service are combined to form a calculation graph,and the capability of realizing AI service output from data is finallyachieved by executing this calculation graph.

In specific applications, the process management module 130 can be usedas a calling system of the AI capability research and developmentplatform, and is the core of link automation of the AI capabilityresearch and development platform. In an example, the process managementmodule 130 can prepare data (such as data required for training models)or models (such as batch prediction) for each tool node in thecalculation graph, and then allocate a calculation resource (such as a“hadoop” calculation resource) to it. And after the tool is executed,its output is registered to a corresponding output resource (such as atraining tool output model).

In practical applications, the process management module 130 can providea sketchpad page. The user can complete the construction of a workflowby manners such as dragging data and tools in the sketchpad page, andthen execute the workflow to obtain the required output, such asinputting data in the workflow and outputting an updated service afterrunning.

It can be understood that in practical applications, the processmanagement module 130 can also be set to support a reusable workflow. Inan example, a logo recognition service process is a relativelycomplicated workflow; after this recognition service process isconstructed in the process management module 130, if the new businessparty needs to perform additional picture recognition, etc., only theinput and part of the configuration of the workflow needs to beadjusted, then the required model and service can be produced, whichprovides the possibility of rapid migration for applying to new similarproblems. The embodiment of the present disclosure does not specificallylimit this.

In an example, FIG. 5 is a schematic diagram of a sketchpad interface ofthe process management module 130 according to an embodiment of thepresent disclosure. Any end-to-end AI research and developmentvisualization process can be completed by dragging various tools on thesketchpad. The left side of the sketchpad is a resource box (input andoutput placeholders; tool box), the middle is the drag area, and theright side is a corresponding tool or resource configuration area. Theuser can drag the input, multiple tools, output boxes in order andcomplete the connecting lines based on upstream and downstreamdependency relationship to generate any end-to-end research anddevelopment process.

The connecting line between tools indicates a process dependency, whichdetermines the execution order between tools, and requires a downstreamtool to start execution after all upstream tools have been executed. Theconnecting line from a tool groove to a next tool groove means that thelatter depends on the former in the process. Parameter dependency notonly meets the tool execution order required in the process dependency,but also specifies the correspondence between upstream and downstreamparameters. The upstream output parameters are directly passed to thedownstream input parameters, and the AI platform automatically completesthe application and transfer of intermediate resources. When clicking astatus button above the connecting line, the corresponding relationshipbetween the upstream and downstream parameters can be seen, and can beadjusted according to needs.

After constructing or updating the workflow, a save workflow button isneeded to click to save the workflow “Graph”, various tools and resourceconfiguration items. The tools and resource configuration items arecalled global configuration of the workflow. The global configuration ofthe workflow refers to the collection of the configuration of the stepsof the workflow saved by the user, and can be loaded through thedrop-down selection. The running of the workflow supports periodicscheduling, which facilitates routine running of processes. Afterfilling in the workflow configuration, click a run button to startscheduling, fill in the periodic scheduling information, and click OK.

In the embodiment of the present disclosure, the model management module140 can also become a model warehouse, and the model management module140 can store models produced by various training tasks. Specifically, amodel can include model weight itself and key information in the modelproduction process, such as, the creator, training data set information,model index, and so on, so as to help the user understand the model morecomprehensively.

In an example, FIG. 6 is a schematic diagram of an interface of themodel management module 140 according to an embodiment of the presentdisclosure. In the interface of the model management module 140, thename, creation time, creator, status and the like of each model can alsobe displayed, which is not specifically limited in the embodiment of thepresent disclosure.

It can be understood that in practical applications, the models in themodel management module 140 can also be set to be used for servicelaunch, batch prediction, model fine-tuning, etc., which is notspecifically limited in the embodiment of the present disclosure.

Embodiments of the present disclosure provide an AI capability researchand development platform and a method, where the AI capability researchand development platform can support the online process from datacollection to model acquisition, and can efficiently perform modeldevelopment. Specifically, the AI capability research and developmentplatform of an embodiment of the present disclosure includes: a datamanagement module, a tool management module, a process management moduleand a model management module, where the data management module isconfigured to perform data processing on received data, where the dataprocessing includes at least one of the following: analyzing the datatype of the data, converting the data according to a preset data formatand storing the data; the tool management module is configured to storeat least one tool, each tool being used to execute a preset processingflow; the process management module is configured to perform modeltraining according to the tool provided by the tool management moduleand the data provided by the data management module; the modelmanagement module is configured to store the model obtained by the modeltraining. During model training in the embodiment of the presentdisclosure, data collection, model training and the like can beuniformly processed on the platform without communication and debuggingby offline personnel from multiple parties, and the developmentefficiency is relatively high.

Optionally, a task module can further be included in the embodiment ofthe present disclosure, as shown in FIG. 7 . FIG. 7 is a schematicdiagram of an interface of the task module according to an embodiment ofthe present disclosure. The task module can be in the form of “Graph” tovisually display the real-time status of each step during the workflowrunning, and the user can click each step to view task details.Optionally, in the AI capability research and development platformaccording to the embodiment of the present disclosure, the datamanagement module 110 is further configured to: collect to-be-back-flowdata, where the to-be-back-flow data is data that meets a presetcondition; the to-be-back-flow data is used to provide source data formodel iteration.

In the embodiment of the present disclosure, the data management module110 can also automatically realize data back-flow. The collectedto-be-back-flow data is automatically recharged to the data set of thedata management module 110 to realize automatically update of the dataset, and in the subsequent model training, the automatic iteration canbe performed according to the updated data, so that the task of datacollection can be realized automatically, which improves the efficiencyof model training.

In specific applications, the to-be-back-flow data may be the case that:when the user performs an operation such as opening a webpage in thedata platform, if there is a picture on the webpage that the platformcurrently cannot recognize during the operation of opening the webpage,the picture can be collected as the to-be-back-flow data. Theto-be-back-flow data may also be the case that: the to-be-back-flow datathat needs to be flowed back is filtered out in the platform pageaccording to a filtering condition preset by the platform. It can beunderstood that the specific content of the to-be-back-flow data can bedetermined according to a practical application scenario, which is notspecifically limited in the embodiment of the present disclosure.

It can be understood that in practical applications, the collectionperiod of the to-be-back-flow data can be set according to a practicalload situation, and the to-be-back-flow data is collected at intervalsof the collection period to balance the load and improve the runningefficiency of the AI capability research and development platform. In anexample, the collection period may be any value from thirty seconds tofive minutes, which is not specifically limited in the embodiment of thepresent disclosure.

In an implementable manner of the embodiment of the present disclosure,the collecting of the to-be-back-flow data includes:

setting the to-be-back-flow data in a back-flow catalog; collating theback-flow catalog according to a preset frequency; where the collatingincludes: sorting to-be-back-flow data which is collected during apreset time period, and setting a same kind of to-be-back-flow data inthe to-be-back-flow data which is collected during the preset timeperiod into one back-flow catalog.

In an example, as shown in FIG. 8 , FIG. 8 shows a schematic diagram ofa back-flow catalog. The AI capability research and development platformcan define a storage file “backflow-data” of the to-be-back-flow data,and collect the to-be-back-flow data regularly. The to-be-back-flow datais set in different back-flow catalogs upon defining different names. Inan example, the data can be divided according to the area, where theto-be-back-flow data collected in area A is stored in “flow-id-0” asshown in FIG. 8 , the to-be-back-flow data collected in area B is storedin “flow-id-1” as shown in FIG. 8 , and the to-be-back-flow datacollected in area C is stored in “flow-id-2” as shown in FIG. 8 . It canbe understood that a back-flow catalog can include multiple levels ofsub-catalogs. As shown in FIG. 8 , the sub-catalogs can be further setaccording to the date and the like, and the to-be-back-flow data iscorrespondingly stored in the sub-catalogs.

In specific applications, since the back-flow data of one data stream isusually divided in many small files, which may generate more spacedebris, the back-flow data can be regularly collated. In an example, thedate before 30 days can be collated every day, and the data belonging tothe same “flow_id” can be collated into one file, which is notspecifically limited in the embodiment of the present disclosure.

Optionally, the tool management module 120 is further configured to:receive, in a tool creation page of the tool management module, a toolcreation operation of the user; and generate a tool according to thetool creation operation.

In the embodiment of the present disclosure, the tool management module120 can provide the tool creation page. The user can perform a toolcreation operation such as inputting code, configuring parameters in thetool creation page according to actual needs, and a custom tool can begenerated based on the tool creation operation, thereby meeting thediverse needs of the user for tools.

Optionally, the data management platform further includes: a testmodule, configured to test the trained model; a platform managementmodule, configured to coordinate and manage the data management module,the tool management module, the process management module, the modelmanagement module and the test module at a project granularity.

In the embodiment of the present disclosure, the test module can includea test tool. In specific applications, when the test module tests amodel, the model can be connected to the test tool in the sketchpad toobtain a test result. Optionally, the test module can also generate atest report, which is convenient for the user to view the test result.

In the embodiment of the present disclosure, considering that theproject granularity is usually used in actual modeling, the platformmanagement module can coordinate and manage the data management module,the tool management module, the process management module, the modelmanagement module and the test module at the project granularity,thereby improving the efficiency of project modeling. It can beunderstood that the platform management module can also be used for userauthentication, user guidance, etc., to assist the user in using theplatform, which is not specifically limited in the embodiment of thepresent disclosure.

In summary, in the AI capability research and development platform ofthe embodiment of the present disclosure, the data management module hasconvenient capabilities of picture data import, export, preview,viewing, conversion and statistics, and can be seamlessly docked withservices such as model training, public test annotating, and data backflow; the model management module can provide unified model storage,management, encryption and automated evaluation of testing and launchingcapabilities; the platform management module can provide unifiedvisual-type-service management, virtualization technology, convenientauthentication, statistics, and data back flow support; the toolmanagement module has easy-to-use functions of tool making, managementand use; the process management module can support convenient processconstruction and automated running capabilities, so that the AIcapability research and development platform of the embodiment of thepresent disclosure can support whole-process automated upgrade anditeration from data to model to service and then to data. When the AIcapability research and development platform is applied to AI researchand development, all links of AI research and development are completedon the platform, and offline communication and confirmation are almostno longer needed, which can greatly improve efficiency of the AIresearch and development.

In order to illustrate the AI capability research and developmentplatform of the embodiment of the present disclosure more clearly, thespecific application process from data collection to model training isdescribed below with the project granularity. It can be understood thatthe project can be a project established in research and development,and the specific content of the project can be determined according to apractical application scenario, which is not specifically limited in theembodiment of the present disclosure.

The data management module 110 is specifically configured to:

receive to-be-processed data of a project; analyze a data type of theto-be-processed data; convert to-be-processed data whose data type meetsa preset condition in the to-be-processed data into target data; wherethe target data has the preset data format; perform statistic on thetarget data; sort the target data into a test set and a data set.

In the embodiment of the present disclosure, the to-be-processed datamay be local data of the user, and the data management module canreceive local data uploaded by the user; the to-be-processed data mayalso be data on a web page, and the data management module can receivedata sent from the web page, which is not specifically limited in theembodiment of the present disclosure.

In the embodiment of the present disclosure, an analyzing control can beset in the data management module 110, and after receiving the triggerof the analyzing control by the user, the data management module 110 cananalyze the data type of the to-be-processed data. In an example, whenanalyzing the data type of the to-be-processed data, the data managementmodule 110 can infer the type of the to-be-processed data and the dataset “schema” (“schema” is a language for describing and standardizingthe logical structure of a file, and the biggest role thereof is toverify the correctness of the logical structure of the file.) etc. Ifthe data type of the to-be-processed data meets a preset condition, theto-be-processed data can be converted to the target data; optionally, ifthere is data whose data type does not meet the preset condition in theto-be-processed data, the to-be-processed data whose data type does notmeet the preset condition in the to-be-processed data can be modifiedinto data of the preset data format according to a modificationoperation of the user, and the modified data is converted to the targetdata. It can be understood that the preset condition can be determinedaccording to a practical application scenario, which is not specificallylimited in the embodiment of the present disclosure.

In practical applications, after analyzing the data type of theto-be-processed data, the to-be-processed data can be identified by thefile name and content. In an example, taking the to-be-processed databeing a picture as an example, the to-be-processed data produced afteranalyzing can include: a filename which can represent a path of thepicture data in a TAR (a compression and packaging tool on Unix andUnix-like systems, which can combine multiple files into one file, withthe suffix of the packed file being also TAR) package; content which canbe picture data. A label name can further be extracted from thefilename. In order to facilitate the subsequent training docking, aclass identity (ID) can be assigned to each label, and the picture andthe ID of the label field are renamed.

In an example, in an implementable manner, the method for converting theto-be-processed data into the target data can be: popping up a codeinput box according to a conversion trigger from the user, andconverting the to-be-processed data into the target data according tocode entered by the user in the code input box. In the conversion, a“schema” editing area can also be provided, and in the “schema” editingarea, the user can modify the ID of a content field to “image”, and addlabel and class_name fields. In specific applications, a trial runningfunction can also be set up; the user can click a trial run button toexecute the code conversion with a small amount of data, preview theconversion result, and after confirming that the result is correct,click a convert button to initiate a data conversion task.

In specific applications, the converted target data may have multipleclasses, so the statistic on the target data can be performed. In anoptional implementation, the performing statistic on the target dataincludes: querying data of at least one preset class in the target data;and performing, in each preset class, statistic on data of the eachpreset class.

In an example, as shown in FIG. 9 , the target data obtained byconverting the to-be-processed data can have four columns: filename,image, label, and class_name, where the label and class_name areclassification information of a picture. The user can query pictures inthe target data according to a class, and can also initiate astatistical task to perform statistic on the distribution of samples.Specifically, the user can query samples according to the class afterturning on a data set query function, and the user can obtain thedistribution of the samples by initiating the statistical task. In anexample, the user can open a statistical-item filtering panel, choosethe label and class_name fields to perform statistic in the form of datatype “enum”, and initiate the statistical task.

In specific applications, a data set is needed in model training, and atest set is needed in model testing. Therefore, in the data processingmodule, the target data can also be divided into the test set and thedata set. In an example, taking the target data being “Caltech 101” asan example, before using “Caltech 101” data for training, the “Caltech101” can be divided into a training set and a test set according to aratio of 80-20. In an implementation, the AI capability research anddevelopment platform can provide a data split button; the user can clickthe data split button, drag a slider to specify the split ratio, fill inthe names of the two data sets produced by the splitting, and finallyclick the data split button to start a splitting task. In an example, asshown in FIG. 10 , after the splitting task is completed, the two datasets “Caltech 101 train” and “Caltech 101 test” produced by thesplitting can be in a completed state.

Optionally, the process management module 120 is further configured to:

provide a model training user interface; receive a target data set and atarget tool chose by the user in the model training user interface;receive, in the model training user interface, a connecting line betweenthe target data set and the target tool from the user; transfer thetarget data set to the target tool according to the connecting line; inthe case of receiving a running instruction, perform model trainingaccording to the target tool to obtain a trained model.

In the embodiment of the present disclosure, the model training userinterface can also be referred to as a sketchpad page.

In specific applications, a tool catalog corresponding to the existingtools in the tool management module can be displayed on the sketchpadpage. The tool catalog mainly includes meta information and entryscripts, where the meta information records basic information of tools,entry execution commands, front-end dynamic configuration and toolidentities, and the entry scripts are main entries of tool servicelogics. By triggering a tool identity in the tool catalog, the tool canbe called.

The existing data identities in the data management module can also bedisplayed on the sketchpad page. By triggering a data identity, the datacan be called.

When performing model training, as shown in FIG. 11 , the user can dragdata and a tool on the left side of the sketchpad to an editing area ofthe sketchpad, and connect the data with the tool to create anexecutable workflow. Taking a sorting training workflow as an example,an input resource can be two data sets: a training data set“train-dataset” and a test data set “test-dataset”, an output resourceis a model, and “caltech101_trainer” can be used as the tool. After atraining task is initiated, the input resource (data) can beautomatically transferred to the tool, and a model file produced by thetraining can be released to the model warehouse.

In specific applications, when executing the workflow, various resourcesand tool items can also be chose in the sketchpad page by drop-downselection, the corresponding parameter configuration is filled in, andthe connecting line status is confirmed as correct. It can be understoodthat if there is an error, operations such as modifying can beperformed.

In specific applications, after completing the above steps, a sortingtraining workflow task is completed. The user can also view the statusof each step and log information on a task list page. Specifically, theuser can jump from the sketchpad page to the task list page, and viewthe details of the steps after finding the task that was initiated. Inan example, a schematic diagram of a log is shown in FIG. 12 , where a“compass log” is a platform execution log link, and a “job log” isgenerally a cluster task log link.

Optionally, the process management module is further configured to:display a model corresponding to a model viewing instruction in the caseof receiving the model viewing instruction.

In the embodiment of the present disclosure, for a trained model, theuser can also view and reuse the model, therefore, in the case ofreceiving the model viewing instruction, the process management modulecan display the model corresponding to the model viewing instruction,which is convenient for the user's subsequent processing of the model.

In summary, the embodiments of the present disclosure provide an AIcapability research and development platform and a method, where the AIcapability research and development platform can support the onlineprocess from data collection to model acquisition, and can efficientlyperform model development. Specifically, the AI capability research anddevelopment platform of an embodiment of the present disclosureincludes: a data management module, a tool management module, a processmanagement module and a model management module, where the datamanagement module is configured to perform data processing on thereceived data, where the data processing includes at least one of thefollowing: analyzing the data type of the data, converting the dataaccording to a preset data format and storing the data; the toolmanagement module is configured to store at least one tool, each toolbeing used to execute a preset processing flow; the process managementmodule is configured to perform model training according to the toolprovided by the tool management module and the data provided by the datamanagement module; the model management module is configured to storethe model obtained by the model training. During model training in theembodiment of the present disclosure, data collection, model trainingand the like can be uniformly processed on the platform withoutcommunication and debugging by offline personnel from multiple parties,and the development efficiency is relatively high.

As shown in FIG. 13 , FIG. 13 is a schematic flowchart of a dataprocessing method according to an embodiment of the present disclosure.Applied to an AI capability research and development platform, themethod can specifically include:

Step S101: performing data processing on received data, where the dataprocessing includes at least one of the following: analyzing a data typeof the data, converting the data according to a preset data format, andstoring the data.

Step S102: performing model training according to a tool and the data.

Step S103: storing a model obtained by the model training.

Optionally, the method also includes:

collecting to-be-back-flow data, where the to-be-back-flow data is datathat meets a preset back-flow condition; the to-be-back-flow data isused to provide source data for model iteration.

Optionally, the collecting to-be-back-flow data includes:

setting the to-be-back-flow data in a back-flow catalog;

collating the back-flow catalog according to a preset frequency; wherethe collating includes: sorting to-be-back-flow data which is collectedduring a preset time period, and setting a same kind of to-be-back-flowdata in the to-be-back-flow data which is collected during the presettime period into one back-flow catalog.

Optionally, the method also includes:

receiving, in a tool creation page, a tool creation operation of a user;

generating a tool according to the tool creation operation.

Optionally, the method also includes:

testing the model obtained by the training.

Optionally, after testing the model obtained by the training, the methodfurther includes:

generating a test report.

Optionally, a type of the tool includes at least one of the following: adata cleaning type, a data mining type, a model training type, a serviceevaluation type, a data back-flow type, and a batch prediction type.

Optionally, the performing data processing on the received dataincludes:

receiving to-be-processed data of a project;

analyzing a data type of the to-be-processed data;

converting to-be-processed data whose data type meets a preset conditionin the to-be-processed data into target data; where the target data hasa preset data format;

performing statistic on the target data;

sorting the target data into a test set and a data set.

Optionally, the method also includes:

modifying, according to a modification operation of the user,to-be-processed data whose data type does not meet the preset conditionin the to-be-processed data into data of the preset data format.

Optionally, the performing statistic on the target data includes:

querying data of at least one preset class in the target data;

performing, in each preset class, statistic on the data of the eachpreset class.

Optionally, the method also includes:

providing a model training user interface;

receiving, in the model training user interface, a target data set and atarget tool selected by the user;

receiving, in the model training user interface, a connecting linebetween the target data set and the target tool from the user;

transferring the target data set to the target tool according to theconnecting line;

performing model training according to the target tool to obtain atrained model in the case of receiving a running instruction.

Optionally, the method also includes:

displaying a model corresponding to a model viewing instruction in thecase of receiving the model viewing instruction.

In summary, the embodiments of the present disclosure provide an AIcapability research and development platform and a method, where the AIcapability research and development platform can support the onlineprocess from data collection to model acquisition, and can efficientlyperform model development. Specifically, the AI capability research anddevelopment platform of an embodiment of the present disclosureincludes: a data management module, a tool management module, a processmanagement module and a model management module, where the datamanagement module is configured to perform data processing on thereceived data, where the data processing includes at least one of thefollowing: analyzing the data type of the data, converting the dataaccording to a preset data format and storing the data; the toolmanagement module is configured to store at least one tool, each toolbeing used to execute a preset processing flow; the process managementmodule is configured to perform model training according to the toolprovided by the tool management module and the data provided by the datamanagement module; the model management module is configured to storethe model obtained by the model training. During model training in theembodiment of the present disclosure, data collection, model trainingand the like can be uniformly processed on the platform withoutcommunication and debugging by offline personnel from multiple parties,and the development efficiency is relatively high.

The data processing method provided by the embodiments of the presentdisclosure can be applied to the methods executed by the modules shownin the corresponding embodiments described above. The implementation andthe principle are the same, and will not be repeated again.

An embodiment of the present disclosure further provides an electronicdevice, including: a processor, a memory and a computer program; wherethe computer program is stored in the memory and configured to beexecuted by the processor, and the computer program includesinstructions for performing the method according to any one of theforegoing embodiments.

An embodiment of the present disclosure further provides acomputer-readable storage medium, having a computer program storedthereon, which, when being executed, implements the method according toany one of the foregoing embodiments.

It can be understood by those skilled in the art: all or part of thesteps of realizing the above method embodiments can be completed byhardware related to program instructions. The foregoing program can bestored in a computer-readable storage medium. When the program isexecuted, the steps involving the foregoing method embodiments areexecuted; and the foregoing storage medium includes various media thatcan store program codes, such as a ROM, a RAM, a magnetic disk, or anoptical disc.

Finally, it should be noted that the above embodiments are only used toillustrate the technical solutions of the present disclosure, but notlimited them; although the present disclosure has been illustrated indetail with reference to the foregoing embodiments, those skilled in theart should understand that: the technical solutions described in theforegoing embodiments may still be modified, or some or all of thetechnical features can be equivalently replaced; and these modificationsor replacements do not deviate the essence of the correspondingtechnical solutions from the scope of the technical solutions of theembodiments of the present disclosure.

What is claimed is:
 1. An artificial intelligence (AI) capabilityresearch and development system, wherein the system comprises: aprocessor; a data management module, configured to execute on theprocessor and perform data processing on received data, wherein the dataprocessing comprises at least one of the following: analyzing a datatype of the data, converting the data according to a preset data format,and storing the data; a tool management module, configured to execute onthe processor and store at least one tool, each of the at least one toolbeing used to execute a preset processing flow; a process managementmodule, configured to execute on the processor and perform modeltraining according to the tool provided by the tool management moduleand the data provided by the data management module; and a modelmanagement module, configured to execute on the processor and store amodel obtained by the model training; wherein the data management moduleis specifically configured to: receive to-be-processed data of aproject; analyze a data type of the to-be-processed data; convertto-be-processed data whose data type meets a preset condition in theto-be-processed data into target data; wherein the target data has thepreset data format; perform statistic on the target data; and sort thetarget data into a test set and a data set; wherein the processmanagement module is further configured to: provide a model traininguser interface; receive, in the model training user interface, a targetdata set from the target data and a target tool selected by a user;receive, in the model training user interface, a connecting line betweenthe target data set and the target tool from the user; transfer thetarget data set to the target tool according to the connecting line; andperform model training according to the target tool to obtain a trainedmodel in a case of receiving a running instruction.
 2. The systemaccording to claim 1, wherein the data management module is furtherconfigured to: collect to-be-back-flow data, wherein the to-be-back-flowdata is data that meets a preset back-flow condition; theto-be-back-flow data is used to provide source data for model iteration.3. The system according to claim 2, wherein collecting to-be-back-flowdata comprises: setting the to-be-back-flow data in a back-flow catalog;and collating the back-flow catalog according to a preset frequency;wherein the collating comprises: sorting to-be-back-flow data which iscollected during a preset time period, and setting a same kind ofto-be-back-flow data in the to-be-back-flow data which is collectedduring the preset time period into one back-flow catalog.
 4. The systemaccording to claim 1, wherein the system further comprises: a testmodule, configured to execute on the processor and test the modelobtained by the training; and a system management module, configured toexecute on the processor and coordinate and manage the data managementmodule, the tool management module, the process management module, themodel management module and the test module at a project granularity. 5.The platform system according to claim 1, wherein the module managementmodule is further configured to store at least one of a creator,training data set information and model index information of the model.6. A data processing method, wherein the method comprises: performingdata processing on received data, wherein the data processing comprisesat least one of the following: analyzing a data type of the data,converting the data according to a preset data format, and storing thedata; performing model training according to a tool and the data; andstoring a model obtained by the model training; wherein the performingdata processing on received data comprises: receiving to-be-processeddata of a project; analyzing a data type of the to-be-processed data;converting to-be-processed data whose data type meets a preset conditionin the to-be-processed data into target data; wherein the target datahas the preset data format; performing statistic on the target data; andsorting the target data into a test set and a data set; wherein themethod further comprises: providing a model training user interface;receiving, in the model training user interface, a target data set fromthe target data and a target tool selected by a user; receiving, in themodel training user interface, a connecting line between the target dataset and the target tool from the user; transferring the target data setto the target tool according to the connecting line; and performingmodel training according to the target tool to obtain a trained model ina case of receiving a running instruction.
 7. The method according toclaim 6, further comprising: collecting to-be-back-flow data, whereinthe to-be-back-flow data is data that meets a preset back-flowcondition; the to-be-back-flow data is used to provide source data formodel iteration.
 8. The method according to claim 7, wherein thecollecting to-be-back-flow data comprises: setting the to-be-back-flowdata in a back-flow catalog; and collating the back-flow catalogaccording to a preset frequency; wherein the collating comprises:sorting to-be-back-flow data which is collected during a preset timeperiod, and setting a same kind of to-be-back-flow data in theto-be-back-flow data which is collected during the preset time periodinto one back-flow catalog.
 9. The method according to claim 6, furthercomprising: receiving, in a tool creation page, a tool creationoperation of a user; and generating a tool according the tool creationoperation.
 10. The method according to claim 6, further comprising:testing the model obtained by the training.
 11. The method according toclaim 10, wherein after testing the model obtained by the training, themethod further comprises: generating a test report.
 12. The methodaccording to claim 6, wherein a type of the tool comprises at least oneof the following: a data cleaning type, a data mining type, a modeltraining type, a service evaluation type, a data back-flow type, and abatch prediction type.
 13. The method according to claim 6, furthercomprising: modifying, according to a modification operation of a user,to-be-processed data whose data type does not meet the preset conditionin the to-be-processed data into data of the preset data format.
 14. Themethod according to claim 6, wherein the performing statistic on thetarget data comprises: querying data of at least one preset class in thetarget data; and performing, in each of the at least one preset class,statistic on data of the each of the at least one preset class.
 15. Themethod according to claim 6, further comprising: displaying a modelcorresponding to a model viewing instruction in a case of receiving themodel viewing instruction.
 16. An electronic device, comprising: aprocessor, a memory, and a computer program; wherein the computerprogram is stored in the memory and configured to be executed by theprocessor, and the computer program comprises instructions for executingthe method according to claim
 6. 17. The electronic device according toclaim 16, the computer program comprises instructions for: collectingto-be-back-flow data, wherein the to-be-back-flow data is data thatmeets a preset back-flow condition; the to-be-back-flow data is used toprovide source data for model iteration.
 18. The electronic deviceaccording to claim 17, wherein the computer program further comprisesinstructions for: setting the to-be-back-flow data in a back-flowcatalog; and collating the back-flow catalog according to a presetfrequency; wherein the collating comprises: sorting to-be-back-flow datawhich is collected during a preset time period, and setting a same kindof to-be-back-flow data in the to-be-back-flow data which is collectedduring the preset time period into one back-flow catalog.
 19. Theelectronic device according to claim 16, the computer program furthercomprises instructions for: receiving, in a tool creation page, a toolcreation operation of a user; and generating a tool according to thetool creation operation.
 20. A non-transitory computer-readable storagemedium, having a computer program stored thereon, which, when beingexecuted, implements the method according to claim 6.